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The stochastic entropy generated during the evolution of a system interacting with an environment 
may be separated into three components, but only two of these have a non-negative mean. The 
third component of entropy production is associated with the relaxation of the system probability 
distribution towards a stationary state and with nonequilibrium constraints within the dynamics 
that break detailed balance. It exists when at least some of the coordinates of the system phase space 
change sign under time reversal, and when the stationary state is asymmetric in these coordinates. 
We illustrate the various components of entropy production, both in detail for particular trajectories 
and in the mean, using simple systems defined on a discrete phase space of spatial and velocity 
coordinates. These models capture features of the drift and diffusion of a particle in a physical 
system, including the processes of injection and removal and the effect of a temperature gradient. 
The examples demonstrate how entropy production in stochastic thermodynamics depends on the 
detail that is included in a model of the dynamics of a process. Entropy production from such a 
perspective is a measure of the failure of such models to meet Loschmidt's expectation of dynamic 
reversibility. 



I. INTRODUCTION 



Entropy production is a measure of the irreversibil- 
ity of a thermodynamic process: the difficulty, even 
impossibility of reversing the observed, often macro- 
scopic behaviour of a system that exchanges heat or 
matter with a complex environment. The apparent 
breakage of time reversal symmetry associated with 
thermodynamic irreversibility has attracted discus- 
sion for more than a century. It was given a particu- 
lar focus by Loschmidt's views on Boltzmann's work 
in gas dynamics [l| but the issue was apparent far 
earlier in the contrast between Fourier's law of heat 
flow and Newton's laws in mechanics. But in spite of 
such concerns, the concept of entropy generation in 
the thermodynamics of large systems has been devel- 
oped and applied widely, to the extent that equations 
for macroscopic entropy generation and transport as- 
sociated with nonequilibrium hydrodynamic and ther- 
mal flows are available, for example see 

However, a microscopic understanding of the nature 
of entropy and its production has proved elusive, par- 
ticularly with regard to understanding the one-way 
character of the second law. But some considerable 
steps forward have been made by modelling the micro- 
scopic evolution of a system and its environment using 
a framework of stochastic dynamics [5] and stochastic 
thermodynamics [6-8]. Such an approach explicitly 
breaks time reversal symmetry through the use of a 
simplified model of the interactions between system 
and environment, a consequence of coarse-graining, 
such that a second law can emerge naturally. Never- 
theless, it is imperative to define entropy in this frame- 
work in such a way that makes contact with results 
known to hold macroscopically. Entropy is often inter- 
preted as a measure of uncertainty at the microscopic 
level, and the viewpoint offered by stochastic thermo- 
dynamics is that its overall production is a reflection, 
according to some views verging on the tautological 



Q , of the uncertain or stochastic dynamics that drive 
the evolution, though other interpretations based on 
deterministic dynamics exist as well fTol— Tl2j| . 

The stochastic approach suggests that a change in 
thermodynamic entropy, satisfying the second law, is 
an expectation value, or more simply the mean, of 
the change in a microscopic, path-dependent entropy 
that evolves stochastically in line with the dynamical 
model of the evolution of the system This view 

allows us to consider entropy as an instantaneous mi- 
croscopic property of a system and its environment, 
based on assignments of probabilities to each of the 
available microstates, but with the potential to in- 
crease or decrease during the process in question. This 
confers meaning to the entropy change that follows 
from a single realisation of a process, and avoids as- 
sociating entropy production only with some sort of 
average over many realisations. It is only after such an 
averaging procedure that the total entropy is expected 
to increase. Thus realisations that reduce entropy are 
quite possible: the second law within this framework 
quite clearly holds only in a statistical sense. 

In detail, the entropy change associated with a spe- 
cific evolution of a system, referred to as a trajectory 
or path, is defined in stochastic thermodynamics in 
terms of the probability that the path might be re- 
alised under the prevailing 'forward' dynamics, driven 
by a forward protocol of time dependence, and the 
probability of realisation of a path that represents the 
reversal of the first, under the same dynamical rules 
but driven by a reversed protocol |L3l - tl7j . This defini- 
tion strongly associates the concept of entropy change 
with that of dynamical irreversibility. When the sys- 
tem state is described in a full phase space of spatial 
and velocity variables, or indeed any set that includes 
variables that change sign under time reversal, the 
reversed path clearly corresponds to a set of points 
in phase space that retraces the sequence of spatial 
positions, but with velocity coordinates that are in- 
verted. It should be noted that the sequence where 
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the velocities are not reversed is sometimes inaccessi- 
ble under the given dynamics. The probability of the 
starting configuration for the reversed path is specified 
to be that which is generated from the forward pro- 
cess. Such a specification produces a path-dependent 
entropy change that satisfies a number of requirements 
(see, for example, (T§|1. 

These issues have recently been explored in the con- 
text of a master equation describing the evolution of 
system probabilities over a discrete full phase space 
[19(, and for the stochastic dynamics of continuous 
coordinates that transform differently under time re- 
versal [2(| • It has been established that the total path- 
dependent entropy change for such cases may be di- 
vided into three components, only two of which, to- 
gether with the sum of the three, are expected to be 
non-negative on average. In this paper we analyse 
a number of examples involving a discrete full phase 
space, in order to provide a greater appreciation of this 
division of entropy production, and to reflect on the 
meaning of entropy production when the dynamics of 
a system and its environment are considered at various 
levels of detail under coarse-graining. We begin with 
a summary of the division of path-dependent entropy 
production into its components, before discussing par- 
ticle drift and diffusion, flow brought about by injec- 
tion and removal, and the transport of heat brought 
about through interaction of the particle with a tem- 
perature gradient. We end with some conclusions and 
remarks on how entropy production, defined within a 
framework of stochastic thermodynamics, in essence 
quantifies the failure of Loschmidt's expectation of 
time reversal symmetry and dynamical reversibility 
for a given set of circumstances. 

II. FORMALISM OF ENTROPY 
PRODUCTION 

Consider a particle that can occupy one of L dis- 
crete spatial positions Xi in one dimension, and as- 
sume one of M discrete values of velocity V rn . The 
particle can make stochastic transitions between phase 
space points (Xi, V m ) = (i,m). We define the forward 
dynamics in terms of a transition rate T(i', m'\i, m) 
characterising a move from (i,m) to (i',m!), such 
that the probabilities of occupation of the phase space 
points P(i,m,t) evolve according to the Markovian 
master equation 

dP{l d ^ l) = T(i,m\i',m')P(i',m',t), (1) 

i' ,m' 

where the notation T(i,m\i,m) represents the total 
rate of transitions from point (i,m): 

T(i,m\i,m) = — T(i' ,m'\i,m). (2) 

These dynamical rules can be employed to generate 
stochastic paths of a particle across the discrete phase 



space. Master equation treatments of dynamics in a 
discrete phase space of velocities as well as positions 
have a long history [2l|, [22j . 

Stochastic entropy production is associated with the 
probabilities of dynamically generating a path and its 
reverse, and in a phase space with coordinates that 
change sign under time reversal, such as a velocity, 
the reversal of a path naturally requires an inversion 
of those coordinates. This is indicated in Fig. [TJ where 
we denote a path by the sequence of phase space points 
x = {xq, ■ ■ ■ , Xk, ■ ■ ■ , xn}, each point represented by 
the coordinates (Xi,V m ). A particle resides at point 
Xk in the time interval tk < t < tk+\ such that the 
duration of the path is from to to t^+i- The reversed 
sequence of spatial coordinates and inverted velocity 
coordinates, visited for a reversed sequence of intervals 
and denoted x^ = {ex^, ■ ■ ■ , sxf., ■ ■ ■ ,£x }, where e 
indicates the change in sign of the velocity coordinate 
(such that eXi = Xi and eV m = —V m ), is the appro- 
priate path to use in defining the total entropy change 
associated with path x: 

AS tot (x) = ln(P F (x)/P K (x^)) 1 (3) 

in units of the Boltzmann constant, where P F (x) and 
P K (x' ! ) are the probabilities that the sequences x and 
x^ are generated. Explicitly, the path probability 
P F (x) may be written 

P F (x) = e 5 tN +lT{xN \ XN)dt 'T{x N \x N - 1 )dt N x 

e J ^ T{xN - llxN - l)dt 'T(x N ^\x N ^)dt N ^x (4) 

■■■T(x 1 \x )dt 1 eft T te>M*'P(x ,t ), 

where P(xQ,to) is the probability that the system re- 
sides at Xo at the initial time to, and T(xj + \\xj) 
is synonymous with T(ij+i, rrij-\-i\ij, rrij). The path 
probability is a product of component probabilities of 
residence and transition. P^(x') is constructed simi- 
larly, but the initial distribution is specified to be that 
which generated by all possible paths corresponding to 
the forward process, namely the solution to Eq. ([T]) at 
t = starting from a distribution corresponding 

to P(xo,to). The superscripts refer to the time de- 
pendence of the transition rates: a forward protocol 
of time dependence is signified by an F, and R denotes 
the reversal of this dependence [19]. For the systems 
under consideration in this paper, the distinction has 
no meaning since for simplicity we assume the rates to 
be time- independent, though for clarity the labels are 
retained. The definition of entropy change according 
to Eq. ^ is compatible with other treatments based 
on master equations (l7l . [23| . 

Fig. [1] indicates two further paths that are related 
to x, the probabilities of which are to be established 
using a set of transition rates T ad known as adjoint 
dynamics [TEl |2~H , and given by 

T ad (i' \m'\i,m) = T(i, m\i', m')P st (i', m')/P st (i, m), 

(5) 

where the P st (i,m) are the phase space probabili- 
ties in the stationary state, which is by definition 
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Figure 1. The phase space coordinate sequence x, with 
its associated set of residence times, and its time-reversed 
counterpart x^ , both shown on the left, can be generated 
by the forward dynamics. The probabilities of each are 
used to define the total entropy production AStot of the 
path x. The probabilities of a further two paths x R and 
x T , shown on the right on the same grid, are considered 
according to adjoint dynamics, and serve to define contri- 
butions ASi and AS2- 



the same for both forward and adjoint dynamics. 
The adjoint dynamics are designed to generate a flux 
of probability in the stationary state that is oppo- 
site to that which arises from the forward dynam- 
ics. Path x R = {xn, ■ ■ ■ ,Xq} is the explicit reverse 
sequence of points, without velocity inversion, and 
x T = {exo, ■ ■ ■ ,£Xn} is the forward sequence of spa- 
tial points but with inverted velocities. These paths 
allow us to define path-dependent entropy change-like 
quantities 



ASi(a:) = In (P F (x)/P adR (x R )) 



(6) 



where P ad ' R (a;' R ) is the probability of generating the 
path x R under a reversed protocol of time dependence 
of the transition rates of adjoint dynamics, and 



AS 2 (x) = In (P F (x)/P ad ' F (a; T )) 



(7) 



in a similar notation, where P ad,F (x T ) is the prob- 
ability of generating the path x T under adjoint dy- 
namics with a forward protocol of time dependence. 
These quantities each satisfy an integral fluctuation 
relation (exp(— AiS to t,i,2)) F = 1) where the brackets 
and superscript denote an average over all paths gen- 
erated by the forward dynamics from a given distri- 
bution of initial coordinates, which implies that the 
expectation values of AStot, A«Si and AS2 over the 
forward dynamics are never negative (lH fit! l2~j - [2l| . 
Furthermore, for paths taken by a system when in a 
stationary state, A«Si is identically zero: it then con- 
sists of a sum of changes to system and environmental 
or medium entropies that cancel |19l . |26j . 

The forms taken by ASi , A1S2 and AStot imply that 
we can represent the total entropy change as a sum of 
three components: AiS t ot = ASi + AS2 + AS3. In 
contrast to the other terms, the path average of the 
contribution 



does not have a fixed sign: however the average of this 
quantity in a stationary state is zero. 

Note that the total entropy production in stochastic 
thermodynamics may also be divided into the compo- 
nents 



A&„t = AS, 



sys 



AS, 



mcd 5 



(9) 



corresponding to changes in the entropies of the sys- 
tem and surrounding medium, respectively [(| 0], 
and furthermore that the latter can be divided into 
'housekeeping' and 'excess' contributions AS mc d = 
A<5>hk + AiS cx according to the descriptive scheme for 
nonequilibrium processes proposed by Oono and Pan- 
iconi [27]. The correspondence with the present divi- 
sion of entropy change is ASi — AS sys + A<S CX and 
A6>2 + AS3 = AShk ■ The terms AS2 and AS3 may be 
designated the generalised and transient housekeep- 
ing contributions to entropy production, respectively 
[19j . ASi and AS2 can also be mapped, respectively, 
onto the nonadiabatic and adiabatic en trop y produc- 
tions of Esposito and Van den Broeck [16j . but only 
for systems described by coordinates that are even un- 
der time reversal, or for which the stationary state is 
symmetric in odd variables. 

We can write ASi in terms of the time dependent 
phase space probabilities P(i,m,t), more compactly 
denoted P(xk,t), that satisfy the master equation ([IJ 
and an appropriate initial condition, together with 
the corresponding distribution in the stationary state, 
P st (xk), as follows: 



ASjx) =Y In V , + Vln ^ ' ^C^H • 

U n^-tj+i) £s p{x J+ll t 3+l )p^{ Xj ) 

(10) 

It is useful to associate each term with features of 
the path. The first sum represents contributions due 
to residence at Xj in the periods tj < t < tj+i and 
the second relates to transitions from Xj to ajj+i at 
time tj + i. Similarly, we can write A1S2 in terms of 
the transition rates T and the stationary probabilities 
P st (x k ): 

N 

AS 2 (x) = (tj+i - tj) (Tixjfa) - T{e Xj \e Xj )) 



3=0 

X 1 P st (ex 3 ) T(x j+1 \ Xj ) 



^ P st (ex i+ i) T(exj\ex j+1 ) 



(11) 



Again, the first sum may be viewed as contributions 
due to residence at point Xj for a period tj + i — tj, 
and the second sum may be associated with the tran- 
sitions. Finally, we write 



iV-l 



AS 3 (x)= Vl* (12) 



which is a sum of contributions associated with the 
transitions. Clearly AS3 vanishes if none of the phase 
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space coordinates change sign under time reversal 
(exj — Xj) or if the stationary state is symmetric 
in odd variables (P st (xj) = P st (eXj)). 

The entropy production associated with a specific 
path is straightforward to compute from the elemen- 
tary contributions arising from residence and transi- 
tion. The path x in Fig. [TJ for example, consists of 
residence at four points and three transitions between 
them: both ASi and AS2 are therefore sums of seven 
terms, while AS3 is a sum of only three. 

The meaning of the above formalism is explored 
next using simple stochastic systems that involve ve- 
locity as well as spatial coordinates on a discrete grid. 
The systems are easier to analyse than those described 
by dynamics in continuous coordinates, and so we can 
readily calculate contributions to entropy production, 
both for individual paths and when averaged over all 
possible paths, and understand better their distinct 
physical origins, properties and meaning. 



T(X m +\X,+) = c+a AS 2 = Jnf^±^j AS, =0 




Figure 2. Contributions to path-dependent housekeeping- 
type entropy production in an asymmetric telegraph or 
Kac process. A particle follows a random walk in one 
spatial dimension with periodic boundary conditions, on a 
phase space defined in terms of position Xi and direction 
of motion ±. The transition rates T in the appropriate 
master equation are shown, together with values of A<S2 
and A1S3 specific to each step. 



III. ASYMMETRIC TELEGRAPH PROCESS 

The telegraph or Kac process is a well known exam- 
ple of stochastic dynamics on a phase space involving 
both position and velocity [28]. For our purposes, it 
is best understood as an elaboration, in the follow- 
ing way, of a standard asymmetric random walk in 
one dimension. At times separated by a fixed inter- 
val of length At, a particle chooses the direction of its 
next step to be to the right or left with probabilities 
(c + a) /2c and (c — a) /2c, respectively, with c > 
and — c < a < c. It then spends the subsequent in- 
terval At performing the move at a constant speed. 
The position and direction of motion of the particle 
are recorded halfway through each timestep. At the 
next recording, the particle will have either returned 
to its previous position but with the opposite velocity, 
having reversed its direction of motion in between, or 
will have moved to an adjacent spatial position with- 
out having changed its direction. These events may be 
modelled using a master equation of the form given in 
Eq. ([TJ) evolving the probabilities P(i,±,t) that the 
particle should assume position Xi and velocity ± at 
time t, and with the rates T{Xi + i + |A^+) = c + a, 
T(Xi - \Xi+) = c-a, T(Xi_i - \Xi-) = c-a and 
T(X l + \Xi~) = c + a, namely 



dP(i,+,t) 
dt 



dP(i,-,t) 
dt 



=(c+a) (P(i-l,+,t)+P(i,-,t)) 
-2cP{i,+,t) 

={c-a) (P(i + l,-,t) + P(i,+,t)) 
-2cP(i,-t). 



(13) 

We assume spatially periodic boundary conditions, 
such that the stationary phase space probabilities are 
P st (i,±) = (c ± a)/{2cL). Notice that a/c plays the 
role of the dimensionless nonequilibrium constraint 
parameter in this example: if it were zero, then 



the stationary state would be symmetric in veloc- 
ity and detailed balance in the sense of T(Xi + i + 
|A^+)P st (i, +) = T(Xi ~ \X z+1 -)P st {i + 1, -) would 
hold. 

We have all the ingredients needed to compute the 
entropy production. A section of the phase space and 
the contributions to the housekeeping-type entropy 
changes A^2 and AS3 made by each of the four dis- 
tinct transitions are shown in Fig. [2l There are no 
contributions to AS2 from residence at each site in this 
example, since T{X l + = T(X t - \X t -) = -2c. 

The contributions to A6>i depend on the P(i, ±, t) and 
are not shown in the diagram for reasons of clarity. 

For a given path followed by a particle across the 
phase space, it is clear from the diagram that the con- 
tribution to A6>3 depends solely on whether the veloc- 
ity has changed sign. The distribution of A1S3 over an 
ensemble of paths generated by the dynamics is there- 
fore trimodal on the values ±2 ln[(c + a)/(c— a)} and 
zero. In contrast, the distribution of A^2 depends on 
the net spatial displacement from the initial position, 
as well as whether the velocity has changed sign, and is 
broader, but still discrete in units of ln[(c+a)/(c — a)]. 

To illustrate this, let us introduce test case A: evolu- 
tion from an initial distribution with P(K, +, 0) = 1, 
for a specified K, and P(i,±,0) = for all other 
points. From consideration of the changes to A1S2 as- 
sociated with each transition in Fig. [3J we can deduce 
that if the particle is found at point Xx+k+ at time t, 
having reached it by any path with a winding number 
around the periodic boundaries of zero, then the accu- 
mulated AS2 is equal to k ln[(c+a)/(c — a)]. Similarly, 
if the particle has reached point Xk+r— the accumu- 
lated A1S2 is (fc— 1) ln[(c+a)/(c— a)]. The probabilities 
P(K + k, ±, t) are straightforward to obtain from the 
master equations (jl3[) . and for early times are domi- 
nated by contributions with zero winding number. As 
an illustration, we solve Eqs. (|13p numerically with 
c = 2, a = 1, L = 10 and elapsed time t = 0.3, for 
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A<S 2 /ln3 



Figure 3. Probability distribution of values of A.S2, in 
units of In 3, for test case A: a stochastic process on the 
phase space in Fig. [5] starting from a specific site with 
positive velocity, and for parameters c = 2, a = 1 and 
t = 0.3. 



the given initial condition, and provide a histogram 
of the resulting distribution P(AS2) in Fig. [31 The 
distribution can be used to verify the expected inte- 
gral fluctuation relation ^ exp(— AiS 2 )-P(A.S 2 ) = 1. 
Furthermore, it satisfies a detailed fluctuation rela- 
tion P{AS 2 ) = exp(A<S 2 )P(-A<S 2 ). In contrast, the 
distribution of AS3 for test case A is P(A<S3 = 0) = 
0.825, P(AS 3 = 2 In 3) = 0.175, corresponding to the 
summed probabilities of positive and negative velocity 
states, respectively, for these conditions, and does not 
satisfy either fluctuation relation. 

An illustration of the stochastic evolution of AS± 
may be obtained by considering a transition between 
stationary states brought about by a change in sign of 
the parameter a (from negative to positive) at t = 0. 
The master equations are readily solved to give 

P(i,±,i) = ^T^exp(-2ci), (14) 

and using these we can identify the contributions to 
A1S1 that arise from residence and transitions between 
phase space points: 



A«Si(X 2 + 
ASx(Xi- 
A«Si(X 2 + 



X,, 



In 



(c-a) (c + a(l-2e- 2c *)) 



(c + a) (c — a (1 
-> Xi+) = -ASx{Xi+ -> X t -) 

1 - 2e- 2 ' ; 



Xi+, At) = In ■ 



a 1 



2e~ 2ct )) 
)) 



ASxiXi- ->Xi-,At) =ln- 



- 2e- 2c (*+ A *))) 
a(l-2e- 2ct )) 



all 



2e -2 C (t+At))) ' 

(15) 

where At is the residence period. There are no con- 
tributions for transitions without a change in direc- 
tion. These supplement the assignments given in Fig. 
[5] and using paths generated by Monte Carlo simula- 
tion, can be employed to obtain probability distribu- 
tions P(AiSi), P(AiS 2 ) and P(ASs) associated with 
such an evolution away from a stationary state. We 
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Figure 4. Probability distributions of the entropy pro- 
duction A<Si, A1S2 and A1S3 accumulated in the period 
< t < 0.5 for test case B: evolution away from a station- 
ary state of the asymmetric telegraph process for t < 
with a = — 1 and c = 2, induced by setting a — 1 for 
t > 0. The entropy production is given in units of In 3. 
Note that the distribution for A<Si is continuous whilst 
those for AS2 and AS3 are discrete. 



explore this by introducing test case B with c = 2, 
a = — 1 for t < and a = 1 for t > 0, and illustrate 
the entropy generating behaviour in Fig. 01 Prob- 
ability distributions are determined for elapsed time 
t = 0.5 and can be used to verify that P(ASi) and 
P(A5 2 ) satisfy an integral fluctuation relation. Fur- 
thermore it may be shown that P(A5 2 ) satisfies a 
detailed fluctuation relation. 



Next we consider the mean value of each component 
of entropy production. Clearly, if a ^ there is a 
nonequilibrium stationary state with an overall prob- 
ability current directed spatially to right or left de- 
pending on the sign of a. The stationary state is then 
asymmetric in the velocity coordinate and breaks de- 
tailed balance. The mean rate of entropy production 
d(A<S 2 )/ dt is a sum of transition-specific contributions 
weighted by the phase space probabilities and transi- 
tion rates, and with reference to Fig. [5] it is given 
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by 



d(AS 2 ) 
dt 



= ^P(i ) +,t)((c + a)ln(^ 
i=i ^ \c a 



+(c — a) In 

V c + a 

+ £>(*,-,*) f(c-a)ln 
c + a 



c — a 
c + a 

c + a 



(16) 



+(c + a) In = 2a In 

\c — aj J \c — a, 

Note that there are no terms associated with resi- 
dence. The form of this expression indicates that 
d{AS2)/dt is never negative, consistent with the inte- 
gral fluctuation relation. It is also time independent, 
whether the system is in the stationary state or re- 
laxing towards it, and its magnitude depends on the 
condition for the breakage of detailed balance o / 0. 
We deduce that (AS 2 ) = 2aln[(c + a)/(c — a)]t and 
this result may be confirmed explicitly for the distri- 
bution in Fig. [3] for test case A, and the distribution 
in Fig. [4] for test case B. 

The rate d(AS3) / dt may be computed similarly: 

d(AS 3 ) L 



dt 



= 5^P(i,+,t) -2(c-a)ln 



L 



In 



c + 


a) 


c + 




c — 









( 17 ) 

This expression has no bounds on its sign, but it 
vanishes in the stationary state when P(i,±,i) = 
P st (i,±) = (c±a)/(2cL). It can be integrated for the 
conditions of test case A to obtain (A1S3) = 0.35 In 3 
at t — 0.3, which is compatible with the distribution 
of A1S3 reported earlier. And for the evolution away 
from a stationary state caused by switching the sign 
of a, we can insert Eq. (|14|) into Eq. (fl7|) to obtain 

d(AS 3 ) 



dt 



-4a exp(— 2ct) In 



c + a 



c — a 



(18) 



Finally we consider the increment d(ASi) over an in- 
crement in time St using Eq. (1101) : 

L 

5(AS 1 )=J2 p ( i > +> ~ a)5tAS!(Xi+ -> X,-) 

i=l 

L 

+J2 p (h-Mc + a)StAS 1 (X i - -> X t +) 

i=l 
L 

+ ^P(i, +, t)(l - 2«5t)A5i(X i + Xi+, St) 
i=i 

L 

+ J2 P ^ ~ ~ 2c<5t)A5i(X i - -)• X,-, ft). 

i=l 

(19) 
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Figure 5. Mean rates of entropy production against time 
for test case B: a transition between nonequilibrium sta- 
tionary states of the asymmetric telegraph process. The 
initial state is characterised by c = 2 and a = — 1 and 
a total mean rate of entropy production equal to 2 In 3. 
At t — the parameter a changes instantaneously to +1, 
and the mean rate of change of A<Si (thin solid line), A.S2 
(dashed) and AS3 (dotted) evolve as shown. The mean 
rate of total entropy production (thick solid line) decreases 
temporarily but is eventually restored to its initial value. 



Using the P(i, ±,t) from Eq. (fT4")) for the transition 
between stationary states, and considering terms to 
first order in St, we obtain d(AS\) / dt for such situa- 
tions: 



dt 



: 2a exp(— 2ct) In 



l + 2aexp(-2c£)/(c-a) 
1 



2a exp(— 2ct) /(c + a), 
(20) 

The three mean contributions together with their sum 
d(AStot) / dt are shown in Fig. [5] for test case B. The 
two transient contributions d(AS±)/dt and d(ASs)/dt 
cancel initially, but their differing time dependence 
apparent in Eqs. (|20[) and (fT5)) gives rise to a short- 
lived reduction in the mean rate of total entropy pro- 
duction as the transition proceeds. The system re- 
organises itself until the initial mean rate of entropy 
production is restored. The integrals of the curves be- 
tween t = and £ — 0.5 correspond to the means of 
the distributions in Fig. 01 

It is instructive to contrast this behaviour with the 
mean entropy production associated with the asym- 
metric random walk when described in terms of spa- 
tial position alone. An appropriate master equation 
for probabilities P(i,t) on a spatial grid with peri- 
odic boundaries can be constructed using transition 
rates to right and left of (c + a) and (c — a) respec- 
tively. The stationary probabilities are P st (i) = L^ 1 
and a transition between stationary states brought 
about by a change in sign of a does not disturb them. 
Hence d(AS\)/dt and d(ASz)/dt are zero and the to- 
tal mean rate of entropy production corresponds to 
d{AS%) /dt, which arises from contributions to AS2 
of ln[(c + a)/(c — a)] for a step to the right, and of 
ln[(c — a)/(c + a)] for a step to the left. The mean 
rate of total entropy production in a stationary state 
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is hence 



d(AS tot ) _ d{AS 2 ) 



dt 



dt 



»=i v v ' 



# o^o^o^o^o-»o-*o-k>*o-»o 

Zl U n n u n u n n u n** 
^,o<-o<-o<-o<-o<-o<-o<-o^-o^-o^ 



+P st (i){c-a) In 



c — a 



c + a 



= 2a In 



(21) 

as before and is independent of time. Clearly the 
transition between stationary states is not now ac- 
companied by a change in the mean rate of total en- 
tropy production. The reorganisation of probability 
with respect to the velocity coordinate is neglected 
at this level of description of the dynamics, and the 
time dependence in the mean rate of total entropy 
production seen in the more detailed description in 
Fig. [5] cannot be appreciated. The example serves to 
demonstrate that entropy production depends upon 
the chosen level of coarse graining: it is after all a 
measure of the irreversibility of a dynamical system, 
and perceived irreversibility will depend on the detail 
employed in modelling the dynamics. 



IV. DIFFUSION AND BARRIER CROSSING 

We elaborate the telegraph process now to illus- 
trate the generation of entropy associated with the 
diffusion of a particle across the spatial extent of a 
system driven by a chemical potential gradient. We 
consider the asymmetric random walk of a particle, as 
represented by the telegraph process, but with a = 
for now, and for a system with reflective boundaries. 
The latter condition implies that the probability of a 
change in direction is unity when a particle is inci- 
dent upon a boundary, such that we should modify 
the transition rates in those circumstances. This is 
illustrated in the upper part of Fig. for a system 
with L = 10 spatial points. Transitions occurring at a 
rate c are indicated with standard size single-headed 
arrows. The double-headed arrows at positions i = 1 
and 10 represent reflections at the boundaries and cor- 
respond to transition rates of 2c. In the absence of fur- 
ther processes, it is clear from counting the rates of 
gain and loss at each point that the stationary phase 
space probabilities for such a system are uniform both 
in position and velocity: an equilibrium state corre- 
sponding to zero mean entropy production. 

Now consider transfers of a particle into and out of 
the system. Physically, we can imagine the insertion 
and removal taking place at the spatial boundaries to 
the left and right of the diagram. In order to include 
the situation where the physical system is empty, we 
create an additional phase space point available to the 
particle, but without spatial or velocity coordinates. 
We allow the insertion of a particle from this point into 
physical phase space points corresponding to inwardly 
directed motion at the extreme left and right hand 
positions, as shown by block arrows in Fig. \§\ The 
rates of insertion are zl and Zr, respectively, and such 
rates appear in the relevant master equations multi- 
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Figure 6. A simple model of the dynamics of particle flow 
driven by a chemical potential gradient, and its consequent 
entropy production. The upper part of the diagram shows 
a phase space consisting of ten spatial points each asso- 
ciated with two velocities, with transitions between them 
indicated by various arrows described in the text. Parti- 
cles are injected and removed from the left and right hand 
sites at specific rates. In the lower part of the diagram, a 
section of the phase space with transition rates that rep- 
resent a barrier to the flow is illustrated. 



plied by the probability Pe(£) that the system should 
be empty. To balance these insertions, we include re- 
movals from the extreme left and right hand positions, 
this time from the phase space point corresponding to 
outwardly directed motion, again shown by block ar- 
rows in Fig. El The rates of removal are z£ and 
z^ 1 , respectively, such that an equilibrium state can 
be established at zl = zr = z, with uniform sta- 
tionary probabilities defined by P st (i,m) — Pp/(2L). 
The probability Pp that there is a particle some- 
where in the physical phase space when at equilib- 
rium, divided by the equilibrium probability P|, that 
the system is empty, is then controlled by z accord- 
ing to P F /Pp — 2Lz 2 . The insertion and removal 
rates can be related to the exponential of the chemi- 
cal potential of a local particle bath and the approach 
resembles grand canonical Monte Carlo. Thus if we 
were to consider the stochastic dynamics under condi- 
tions zl > zr, we would expect the particle to follow 
paths involving transfers into and out of the physi- 
cal system, producing a mean flux of probability from 
left to right across the physical phase space, and a 
positive mean rate of entropy production. The ra- 
tio [zl — zr)/ zr plays the role of the dimensionless 
nonequilibrium constraint parameter for this case. 

In order to explore this in detail, we consider the 
nonequilibrium stationary state corresponding to the 
rates c = 1, Zl = 2 and zr = 1. The stationary 
probabilities P st (i,±) are shown as vertical bars in 
the upper part of Fig. [7] together with points repre- 
senting the increments in AS2 associated with steps 
to the right starting from position i. Profiles of path- 
dependent entropy increments associated with steps 
to the left, and with reversals of direction, may also 
be generated but are not shown. The increasing path- 
dependent contributions to entropy production as a 
function of position are a consequence of the constant 
probability current to the right, and the linear de- 
crease in the stationary probabilities. The situation 
is analogous to a stationary state of particle diffusion 
between a source at high chemical potential on the left 
and a sink at low chemical potential on the right. 
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Figure 7. The left and right hand columns at each spatial 
position represent the stationary probabilities for each ve- 
locity in the phase space illustrated in Fig. [6] The upper 
plot corresponds to dynamical conditions c = 1, Zi, = 2 
and zr = 1, whilst the lower plot includes the barrier sec- 
tion around position 5 with a — —0.9. The square symbols 
denote the path-dependent contribution AS2 associated 
with a step to the right from each spatial position (right 
hand axis). Passage across the barrier is characterised by 
a relatively large local value of A<S2- 



This behaviour is modified if the transition rates 
are altered in the fashion indicated in the lower part of 
Fig. [6l The thicker and thinner arrows represent tran- 
sition rates c—a and c+a respectively, with a < 0, and 
we can imagine the replacement of the central part of 
the transition rate scheme by such a 'barrier section'. 
The replacement creates local external forces that im- 
pede the particle flow between source and sink, and 
distort the stationary probability distribution across 
the phase space. Nevertheless, the probability distri- 
bution with zl — Zr even in the presence of the barrier 
remains symmetric in the velocity coordinate and is 
an equilibrium state. For a nonequilibrium stationary 
state with Zl = 2, zr = 1, c = 1 and a = —0.9 (in 
the barrier section) the profile of probabilities across 
phase space and the contributions to AS2 associated 
with each spatial step to the right are illustrated in the 
lower part of Fig. [7] The interpretation is that the 
specific passage of the particle across the barrier is as- 
sociated with a relatively large contribution to entropy 
production. Rare moves in a direction favoured by the 
prevailing thermodynamic forces are more irreversible 
than commonplace ones: this is an intuitively use- 
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Figure 8. Evolution of the mean rates of change of 
A<Si (thin solid line), A<S2 (dashed), A<S3 (dotted) and 
their sum (thick solid line) corresponding to the transition 
brought about by the change in zl from 1 to 2 at t = for 
the system illustrated in the upper part of Fig. [6] zr = 1 
throughout so the mean rates for t < are all zero. 



ful viewpoint, though we recognise it to be somewhat 
tautological Q. The history of entropy production 
associated with a stochastic path taken by a particle 
on this phase space will contain periods during which 
the entropy fluctuates up and down, as the particle 
explores the phase space region to the left of the bar- 
rier, which are terminated by relatively large positive 
spikes in entropy production when the particle crosses 
the barrier. 

The mean rates of change of the various compo- 
nents of entropy production for the system without 
a barrier and with c = 1, for a process starting in 
equilibrium at zl = zr = 1 for t < and driven out 
of equilibrium by an instantaneous switch to zl = 2 
for t > 0, are shown in Fig. [5J The system responds 
to the implied change in chemical potential of the left 
hand source with mean entropy production in all three 
components. For t < the mean rate of production 
of entropy is zero for all components, and so there is a 
discontinuity in the rate of production when the pro- 
cess begins. Once again, this pattern of mean entropy 
production differs with respect to a model of the sys- 
tem that involves only the spatial phase space points: 
in particular there would be no contribution /S.S3 in 
the latter case. At such a coarser level of description 
the reorganisation of the probability distribution over 
the velocity coordinate of phase space would not be 
perceived, and consequently the assessment of the ir- 
reversibility of the process would not be the same. 



V. DYNAMICS ON A RING WITH M 
DISCRETE VELOCITIES 

We now consider a phase space composed of dis- 
crete spatial positions Xi with periodic boundary con- 
ditions, as before, but now with an extended set of 
available velocities V m = (m — 1) — (M — l)/2, where 
m = 1, • • ■ , M with M an even integer. We define 
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Figure 9. A phase space of discrete spatial (labelled i) 
and velocity (labelled m) coordinates with a selection of 
allowed moves of a particle as described in Eqs. (I22p . Pe- 
riodic boundary conditions apply spatially. 



M —m 

T(i+MZ,m+m,m) = A 

M-l 

AS, =ln(A/B) AS 3 =-21n(A/B) 



„ M —m in — 1 „ 

T(i,m\i, m) = -C A B 

M-l M-l 

&S ! =,(i,,n)(A-B) (2m - 1 - M) AS,=0 




T(i + Ai n! ,m\i,m) = C 
AS, =0 
AS, = 



7Xi + Ai,,, ,m— 1 1 i,tn) = 

AS, = -ln(A/fl) 
AS, =21n(A/B) 



Figure 10. An illustration of the four types of behaviour 
of a particle in the phase space illustrated in Fig. [9] indi- 
cating the associated transition rates T and housekeeping- 
type contributions AS2 and A1S3 to entropy production. 
The particle resides at phase space point (Xi, V m ) = (i, m) 
for a time t(i, m) prior to a move described by one of the 
rates in Eq. (|22l) . The A<Si take a form that depends on 
the probabilities P(i,m,t) and are specified in Eq. (|10|) . 



non-zero transition rates to be 
T(i + Ai° n ,m\i,m) = C 

T(i + Ai+,m + l\i, m) = [M - m) A/(M - 1) (22) 
T(i + Ai m ,m - m) = (m - 1)B/(M - 1), 

where A, B and C are positive. The spatial displace- 
ments Ai^ are defined by 



metric telegraph process discussed earlier: the speci- 
fication of the transition rates is different. In fact this 
case is a generalisation of the two- velocity system con- 
sidered previously [l9j . and is a discrete version of a 
treatment of driven Brownian motion of a particle on 
a ring (20| . The stationary probability distribution is 



P st {i,m) = 



(m - xyxA/B)™-- 1 

L{M - to)! (to - 1)! 



1 



l-M 



(24) 

which is uniform over spatial coordinates, and is char- 
acterised by a mean velocity equal to (A — B)(M — 
1)/ (2(^4+73)). The dimensionless nonequilibrium con- 
straint parameter is (A — B)/B: if this were zero, the 
stationary distribution would be symmetric in veloc- 
ity. 

The transition rates and stationary probabilities al- 
low us, as before, to compute contributions to entropy 
production associated with the detailed history of a 
path. Values of A£>2 and AS3 are illustrated for the 
model in Fig. |TD] A6>i is not included since it depends 
on P(i,m,t) and is less compact in form. Note that 
there are now non-zero contributions to A1S2 arising 
from residence at the phase space points. These were 
absent in the telegraph process considered earlier, as 
a consequence of the particular choice of transition 
rates. 

Let us examine the means of entropy contributions 
A1S1, AS2 and AS3 over a short time interval St. We 
average over the behaviour of the particle, employ- 
ing probabilities 1 — T(xj\xj)St for residence over the 
period at xj, and probability T(xj + i\xj)5t for a tran- 
sition from Xj to Xj + i. We do not consider multiple 
transitions during the interval since it is short. Ac- 
cording to Eq. pop the mean increment in A5i is 

S(AS 1 ) = 

E./M-m lf , I P(i,m,i)(M - m)A 
P(i, m, t) — A5t n — — '-^ r — '- 
y ' ''KM -I \ P(i,m + l,t)mB 

P(i,m,t)(m - l)B 



m — 1 
M-l 



-BSt In 



P(j, m - 1, t){M - to + 1)A 



(1 - T(i, m\i, m)5t) ^ — '-^5t 



(25) 



Note that the last term vanishes to order St due to 
normalisation. We rearrange to get 



M-l 



Ai° m = 2(m-1)-(M-1), Ai±=A»°,±l, (23) ^^ = _i_£ £(p(i, m , t )(M - m )A 



and some examples of transitions are illustrated in 
Fig. The dynamics represent changes of velocity 
according to the familiar Ehrenfest model [29l |. such 
that in the limit of large M the stationary velocity dis- 
tribution approximates to a gaussian envelope with a 
mean determined by A and B. The spatial displace- 
ments correspond to propagation at either the velocity 
V m , or an average of V m and V m ±i if there should be 
a change in velocity, for a period At = 2. 

In spite of some similarities in appearance, this 
model with M = 2 is not equivalent to the asym- 



„ x „\ , {P(i,m,t)(M - m)A\ ,„„. 

- P(i, m + 1, t )mB In — — '-^ r '— , (26 

v ' ' ' J \ P(i,m + l,t)mB J ' v ' 

a form that is explicitly positive, or zero in a station- 
ary state when P(i,m,t) = P st {i,m). Similarly 



6(AS 2 ) =J2 p (hm,t) ((A-B 



(2m -l-M) 
' M-l 



M - m , / A\ to - 1 , /j4\ . . 

m^iHb -M-T Bln U))^' !27) 
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for small St, giving 
d{AS 2 ) ^P(i,m,t) 



dt 



^ M-l 1 '\B \B 



+A(M-rn) g-1-lng 



(28) 



which is also explicitly positive unless A — B, the con- 
dition for a symmetric stationary state over velocities, 
in which case it vanishes. Its mean in a stationary 
state is obtained by inserting Eq. (l2~fl : 



d(AS 2 



dt 



(A -BY 
A + B 



(29) 



Finally, 

d(AS 3 ) 
dt 



M - 1 



+2(m-l)Bln(A/B)) 



(30) 



which can take either sign, but which vanishes when 
the system is in the stationary state or when A = 
B. These results with M = 2 coincide with those 
obtained previously (l9| . 

The importance of the velocity coordinate for the 
deeper appreciation of irreversibility in this system 
may be demonstrated by determining the mean en- 
tropy production associated with a transition between 
stationary states brought about by the instantaneous 
swapping of the values of the parameters A and B. 
From a perspective of a treatment in a phase space 
of positions alone, there would be no relaxation of 
the probability distribution: the only entropy produc- 
tion brought about by the transition would arise from 
A1S2 , as was found in test case B for the telegraph pro- 
cess. However, according to Eq. (|29p . which remains 
valid for this coarser grained treatment, the mean rate 
of entropy production would not change. Only from 
a perspective of the more detailed dynamics would 
there be additional contributions AS± and AS 3 , aris- 
ing from the relaxation in the probability distribution 
over velocities. The latter behaviour is illustrated in 
Fig. QT] for an example with parameters A = 2 and 
B = 1 with M — 8, C being arbitrary for this ex- 
ample. The total mean rate of entropy production 
falls momentarily to zero during the transition, as is 
also seen in the treatment of driven Brownian mo- 
tion on a continuous phase space reported elsewhere 
[20]. The deviation from the constant contribution 
d(AS 2 ) /dt is due to the relaxational terms d{ASi)/dt 
and d(ASs)/dt. Without a consideration of velocity 
coordinates, the implication that the system behaves 
less irreversibly, on average, during the transition be- 
tween stationary states would be missed. We conclude 
again that coarse-graining alters the perception of ir- 
reversibility. It is worth noting, however, that the be- 
haviour seen in Fig. [TT] is similar that which emerges 
from a continuum treatment of the same system [20] , 
suggesting that dynamics on a coarse-grained, discrete 
representation of a continuous phase space can still 
capture certain features of the irreversibility. 
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Figure 11. Mean rates of entropy production against time 
for a transition between stationary states for a particle on 
a ring with M = 8 available velocities and transition rules 
illustrated in Fig. [9] and Eq. (f22j) . For t < the system 
is in a stationary state parametrised by A = 1 and 5 = 2. 
For t > 0, these parameter values are swapped and the 
system makes a transition to a new stationary state. 



VI. A SIMPLE MODEL OF THERMAL 
CONDUCTION 



Our final example system is illustrated in Fig. [T2J It 
is one of the simplest phase space dynamical schemes 
that can represent thermal conduction and the asso- 
ciated production of entropy. The phase space con- 
sists of a single spatial position, with four possible 
velocities: fast (suffix /) and slow (suffix s) in each 
direction. The transitions between the points corre- 
spond to the reflection of a particle from boundaries 
situated to the left and right of the spatial position. 
The left hand boundary has the property that slow ar- 
rivals are reflected with probability (b + A6)/(26) into 
the fast returning velocity state and with probability 
(b — Ab)/(2b) into the slow state, with transition rate 
b > and —b < Ab < b. For positive Ab the bias to- 
wards the fast return velocity suggests we can regard 
the left hand boundary as 'hot'. For simplicity, the left 
hand boundary is assumed to partition fast arrivals 
into fast and slow returning states with equal proba- 
bility. In contrast, the right hand boundary reflects 
fast arrivals with probability (b + Ab) / (26) into the 
slow return state, and with probability (6 — Ab)/(2b) 
into the fast state. Slow arrivals at this boundary 
are partitioned with equal probabilities into fast and 
slow returns. These rules allow us to regard the right 
hand boundary as 'cold'. More elaborate schemes 
than these might be conceived, but the symmetries 
in the transitions and associated rates, as illustrated 
in Fig. 1121 simplify the analysis of entropy produc- 
tion (in particular, there are no contributions to AS 2 
from residence at the phase space points) while allow- 
ing the model to capture something of the physics of 
thermal conduction, with Ab representing a thermal 
gradient across the system and Ab/b the dimension- 
less nonequilibrium constraint parameter. 

The contributions to A1S2 and AS3 associated with 
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Figure 12. A simple model of thermal conduction involving 
one spatial point and four values of velocity. A particle is 
reflected from hot and cold boundaries and the consequent 
transitions between fast and slow velocity coordinates in 
each direction are represented by arrows with specific rates 
shown. The contributions AS2 and AS3 to entropy pro- 
duction associated with each transition are given as a func- 
tion of the thermal gradient parameter Ab. 



the eight available transitions in the system are listed 
in Fig. \T2h These are calculated on the basis of 
the transition rates indicated in the diagram and the 
stationary state probabilities P st (Vf) = P st (— V s ) = 
6/(46-2A6) and P st {V s ) = P st {~V f ) = (b-Ab)/(4b- 
2Ab). Since Vf > V s , this implies that the mean par- 
ticle velocity is directed towards the right for Ab > 0. 
Clearly some paths undertaken by the particle be- 
tween the boundaries, such as Vf — > —Vf —> Vf, give 
no net change in entropy, whilst those that involve a 
slowing down at the cold boundary and speeding up 
at the hot boundary (for example Vf — > —V s — > Vf) 
produce a positive increment for Ab > 0. In contrast, 
the circuit — > — Vf — >■ V s produces a negative incre- 
ment, compatible with a distribution of both positive 
and negative values of AiS>2- 

The mean rate of change of AS2 in the stationary 



state may be shown to be 
d(AS 2 ) st 



dt 



P*\Vf)[{b-Ab)\n[l-— I 



A& 



Ab\ 



-[b Ab)\n ^1 + = J j +P st (V s ) (-bin (l + 
-blnil-f)) + P-(-V f )(-bln(l-f 



-bin 1 



Ab 
b 



P st (-V s )[ (6 + Aft) In 1 



-{h- AM In ( 1-^ 



which reduces to 

d(AS 2 ) st _ 2bAb 



dt 



. Ab 
26-A6 ln l 1 + T 



Ab 
~b 

(31) 



(32) 



a form that is never negative for the physical range 
— b < Ab < b. For Ab = we recover the equilibrium 
state where d(AS2) st / dt vanishes and where the phase 
space probabilities are symmetric in velocity. 

The mean value of exp(— AS2) for an incremental 
time period 5t is 

(exp(-A5 2 )) =P(V f ,t) Ub - Ab)St (l - 



-( b + AbhV { 1 + ^pj J +P(V s ,t) fbSt (l + ^ 



^ { l-^)) + P { -Vf, t )(b5t(l-f 



-bSt (l + ^-Jj +P(-V s ,t) l^b + Ab)St (l + ^ 
-(b-Ab)St(l-^\ )+(P(V f ,t) + P(V s ,t) 



+P(-V f ,t)+P(-V s ,t))(l-2bSt), 



(33) 



where the last term is made up of contributions aris- 
ing from residence at each point. This reduces to 
(exp(— AS2)) = 1 for the period St, a result that there- 
fore also holds for a finite time period by consideration 
of a sequence of incremental periods under the Marko- 
vian dynamics. 

By a similar analysis, the mean rate of change of 
A1S3 in the stationary state is 



+2P st (T4)61n (l - ^) + 2P st (-V f )bln (l - ^ 



2P st (-V;)(6- A6) In 1 



Ab 
~b 



0. 



(34) 



vanishing as expected. 

Transient departures from equilibrium can be in- 
vestigated using a numerical solution of the relevant 
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Figure 13. The evolution of the mean rates of entropy pro- 
duction d{ASi)/dt (thin solid line), d(AS2)/dt (dashed), 
d{ASs) / dt (dotted) and their sum d(AStot)/dt (thick solid 
line) for the system shown in Fig. [12] driven by Ab(t) 
specified in Eq. (|35p . d(ASa) /dt goes negative in re- 
sponse to the increase in Ab, and is largely positive when 
Ab decreases. All the other mean rates are non-negative 
throughout since the distributions of those entropy incre- 
ments satisfy integral fluctuation relations. 

master equation. As an example, consider the imposi- 
tion and removal of a thermal gradient brought about 
by the specification 

{i (1 - cos(Trt)) < t < 1 

\ 1 < t < 3 (35) 

i(l + cos(7r(t-3))) 3<<<4 

with Ab — for t < and t > 4, and 6 = 1 
throughout. The mean rates d(ASi)/dt, d(AS2)/dt 
and d(ASs) /dt, together with their sum, are shown 
in Fig. Q2] over the course of such a process. Several 
now-familiar features are apparent. The mean rates of 
change of A«Si, AS2 and AStot, quantities that each 
satisfy integral fluctuation relations, are never nega- 
tive, whilst d(ASa}/ dt can take either sign. d(A6>2) /dt 
and d{ASs)/dt are non-zero only when detailed bal- 
ance is broken through the condition Ab ^ 0. The 
evolution of d(AS2)/dt mirrors the time dependence 
of Ab(t) and both d(AS\) /dt and d(AS^) /dt are tran- 
sient contributions that tend to die away when Ab 
is constant. Of the transient terms, the production 
rate d(AS\)/dt continues to evolve after Ab(t) has 
gone to zero, whilst d(AS^)/dt vanishes in these cir- 
cumstances. The pattern of the mean rate of total 
entropy production d(AStot) / dt provides a character- 
isation of the thermal conduction and its irreversibil- 
ity. It emerges with a delay in response to the time 
dependent thermal gradient, which seems physically 
intuitive. 

The contribution d{AS3)/dt plays an important 
part in establishing this pattern, and it relies on 
the presence of odd velocity coordinates in the phase 
space. In fact it would be impossible to describe the 
entropy production associated with thermal conduc- 
tion without considering the velocity coordinates of 
a system: heat flow is the conveyance of kinetic en- 
ergy. This has already been established through an 



analysis of the process using continuum stochastic dy- 
namics (23 • A heat current with associated entropy 
production cannot be established in a phase space of 
one spatial dimension with reflective boundaries and 
so dynamics on a phase space of spatial points cannot 
capture the irreversibility of heat conduction. This 
is the clearest of our demonstrations that a failure to 
take account of dynamics in full phase space can lead 
to the neglect of an important mechanism of entropy 
production and alter the apparent irreversibility of the 
process in question. 

VII. CONCLUSIONS 

We have investigated several simple examples where 
entropy production can be associated with specific, 
stochastically generated paths taken by a particle on 
a discrete full phase space. We have shown how it 
is formed from three components, each with particu- 
lar statistical properties and a specific relationship to 
the underlying physical origins of irreversibility. The 
component AS± is associated with relaxation towards 
a stationary state. A1S2 is associated with the break- 
age of detailed balance brought about by the dynam- 
ical rules, and with entropy production in nonequilib- 
rium stationary states. A1S3 is also associated with 
the breakage of detailed balance but only exists for 
dynamics that include coordinates that change sign 
(are 'odd') under time reversal 0, |2jj. A further 
condition for its existence is that the probability dis- 
tribution for the stationary state under the prevailing 
conditions has to be asymmetric in an odd coordinate. 
The mean of A1S3 is zero in a stationary state and so, 
like A1S1, it is associated with relaxation. Thus AS3 
has its origin in the dissipative mechanisms that are 
separately responsible for ASi and AiS>2- The three 
components can be related to the changes in system 
and medium entropy @ as well as, in some circum- 
stances, adiabatic and nonadiabatic entropy produc- 
tion Qj|. 

Specifically, we have considered stochastic particle 
dynamics on discrete full phase spaces with transition 
rules that capture aspects of drift and diffusion, bar- 
rier crossing, injection, removal and interaction with a 
thermal gradient. The examples are necessarily sim- 
plified, but they illustrate the properties of the various 
components of entropy production summarised in the 
last paragraph, and furthermore demonstrate that the 
distributions of A«Si and A1S2 satisfy integral fluctu- 
ation relations. They also show that the inclusion of 
additional detail in the specification of a system, or its 
opposite, coarse-graining, can have an impact on the 
assessment of entropy production. This is compatible 
with the notion that entropy is a representation of the 
uncertainty in microscopic state, and that its produc- 
tion is linked to the dynamics employed when mod- 
elling the evolution. By taking into account greater 
levels of detail in a dynamical system, we potentially 
alter our perception of the uncertainty in the micro- 
scopic state, and the pattern of entropy production 
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could be modified result. At the deepest level, 
of course, when we neglect no detail of either system 
or environment, there will be no stochasticity in the 
model (we would then employ the 'real' dynamics) and 
no consequent development of uncertainty, and from 
a point of view of stochastic thermodynamics there 
would be no production of entropy. Entropy change in 
this perspective is subjective, and depends on choices 
made in the modelling. 

We emphasise that these conclusions are based on 
an interpretation of entropy change and the second 
law acquired in the context of stochastic thermody- 
namics. The underlying dynamics have been taken to 
be stochastic and to break time reversal symmetry. 
Entropy change is then associated with dynamical ir- 
reversibility expressed in terms of path probabilities. 
An interpretation based on deterministic and time- 
reversible dynamics is also available, whereby entropy 
generation is associated with the contraction of a con- 
tinuous phase space and it is possible to derive fluc- 
tuation relations and a second law fiol . [Tl| . 

In closing, and from a perspective of stochastic ther- 
modynamics, we reflect briefly on the nature of en- 
tropy and its production. Time reversal invariance 
and determinism imply that events in the future as 
well as the past will be apparent to a being capable of 
perceiving the current state of the universe in all its 
detail, a point made by Laplace. Loschmidt used the 



same reasoning to identify the flaw in Boltzmann's 
initial attempts to build a mechanical model of en- 
tropy production in an isolated ideal gas: the model 
could not provide the arrow of time. But Loschmidt 
is far from being the villain of this piece of the history 
of science [l|. Consideration of time reversal enriches 
the meaning of entropy production in models that ex- 
plicitly break this symmetry, in many cases through 
a representation of the interactions between a system 
and its environment that reflects a state of percep- 
tion inferior to that of Laplace's being. Symmetry 
of equations of motion under time reversal is either 
present or it is not, but the practical consequence of 
its absence emerges on a broader spectrum, and in 
stochastic thermodynamics, entropy production is its 
measure. Some trajectories generated by a stochastic 
model are hard to reverse (meaning the reverse trajec- 
tory is relatively unlikely); some are easier. The sec- 
ond law in stochastic thermodynamics is a statement 
about the statistical dominance of the hard cases. 
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